Fatal or not? Finding errors that lead to dialogue breakdowns in chat-oriented dialogue systems
نویسندگان
چکیده
This paper aims to find errors that lead to dialogue breakdowns in chat-oriented dialogue systems. We collected chat dialogue data, annotated them with dialogue breakdown labels, and collected comments describing the error that led to the breakdown. By mining the comments, we first identified error types. Then, we calculated the correlation between an error type and the degree of dialogue breakdown it incurred, quantifying its impact on dialogue breakdown. This is the first study to quantitatively analyze error types and their effect in chat-oriented dialogue systems.
منابع مشابه
Towards Taxonomy of Errors in Chat-oriented Dialogue Systems
This paper presents a taxonomy of errors in chat-oriented dialogue systems. Compared to human-human conversations and task-oriented dialogues, little is known about the errors made in chat-oriented dialogue systems. Through a data collection of chat dialogues and analyses of dialogue breakdowns, we classified errors and created a taxonomy. Although the proposed taxonomy may not be complete, thi...
متن کاملThe dialogue breakdown detection challenge: Task description, datasets, and evaluation metrics
Dialogue breakdown detection is a promising technique in dialogue systems. To promote the research and development of such a technique, we organized a dialogue breakdown detection challenge where the task is to detect a system’s inappropriate utterances that lead to dialogue breakdowns in chat. This paper describes the design, datasets, and evaluation metrics for the challenge as well as the me...
متن کاملOn Dialogue Breakdown: Annotation and Detection A report from the dialogue breakdown detection challenge
We organized the dialogue breakdown challenge, which is an evaluation workshop dedicated to the detection of breakdowns in Japanese human-machine chat-oriented dialogues. This paper reports analyses made on the breakdown annotation to the evaluation data used in the challenge and on behaviors of the detection systems submitted by six teams. The performances of the ensembles of the systems are a...
متن کاملDialogue Strategies for Improving the Usability of Telephone Human-Machine Communication
Interactions with spoken language systems may present breakdowns that are due to errors in the acoustic decoding of user utterances. Some of these errors have important consequences in reducing the naturalness of human-machine dialogues. In this paper we identify some typologies of recognition errors that cannot be recovered during the syntactico-semantic analysis, but that may be effectively a...
متن کاملبررسی ارتباط بین سیرت اساتید با آموزش مبتنی بر دیالوگ (مطالعه مورد؛ دانشگاه اصفهان)
The present study aims to investigate the relationship between professors' character and dialogue-oriented teaching from the point of view of the students in University of Isfahan using structural equation modeling. This study is an applied one collecting data through descriptive-correlative methods. Statistical population of the study included 13281 students of Isfahan University in the school...
متن کامل